Duet Package: Single Person Analysis Workflow

Duet Package Tutorial: Single Person Analysis

This tutorial demonstrates a complete workflow for analyzing OpenPose data from a single individual using the duet R package.

---

1. Setup and Installation

Install Required Packages

The chunk below will install any packages you are missing. It is set to eval=FALSE so it does not run automatically. Run it manually in the R console if you need to install packages for the first time.

# Install duet from GitHub if you haven't already
# devtools::install_github("themisefth/duet")

# Core packages for this workflow
required_packages <- c(
  "tidyverse", "cowplot", "rempsyc", "magick", "flextable", "remotes")

# Install any missing packages from the list
new_packages <- required_packages[!(required_packages %in% installed.packages()[,"Package"])]
if(length(new_packages)) install.packages(new_packages)

Load Libraries

library(tidyverse)
library(duet)
library(magick)
library(ggplot2)
library(cowplot)
library(rempsyc)
library(flextable)

# Set a consistent theme for ggplot2 plots
theme_set(theme_classic(base_size = 12))

Create Directory Structure

This will create the necessary folders in your project directory to store raw data and outputs.

output_dirs <- c("./openpose_json", "./openpose_csv", "./figures")
for(dir_path in output_dirs) {
  if (!dir.exists(dir_path)) dir.create(dir_path, recursive = TRUE)
}

---

2. Data Conversion: JSON to CSV

The first step is to convert the raw JSON files from OpenPose into a single, tidy CSV file.

Understanding Your Data Structure

Place your raw OpenPose JSON files in a dedicated folder. For this example, we assume they are in ./openpose_json/Individual_01/.

./openpose_json/
├── Individual_01/
│   ├── frame_000001_keypoints.json
│   ├── frame_000002_keypoints.json
│   └── ...
└── ...

Convert JSON to a Single Person CSV

This function reads all JSON files in the specified directory and converts them into one CSV file. We set export_type = 'person' and specify person = 1 to ensure we only process data for the first person detected by OpenPose.

op_create_csv(
  input_path = './openpose_json/Dyad_10',
  output_path = './openpose_csv/Dyad_10',
  model = 'body',
  include_filename = TRUE,
  export_type = 'dyad')

---

3. Data Import and Initial Checks

Import Data

Load the CSV file you just created.

# Read the CSV file into a dataframe
df <- read_csv('./openpose_csv/Dyad_10/Dyad-10.0_B_IDs-B013-B016_body_dyad.csv')

Check Data

It’s good practice to visually inspect a frame to ensure the data has loaded correctly.

# Plot a single frame to check the pose
op_plot_openpose(df, 
                 person = "both",
                 frame_num = 10, 
                 lines = TRUE)

---

4. Data Quality and Preprocessing

Assess Data Quality

The op_plot_quality() function helps you understand the completeness (how many keypoints were detected) and confidence (OpenPose’s certainty) of your data over time.

# First, apply human-readable labels to the keypoints
df_labelled <- op_apply_keypoint_labels(df)

# Plot data completeness
p_completeness <- op_plot_quality(df_labelled, 
                                  plot_type = 'completeness',
                                  threshold_line = 80) # 80% threshold line

# Plot tracking confidence
p_confidence <- op_plot_quality(df_labelled, 
                                plot_type = 'confidence',
                                threshold_line = 80) # Confidence threshold of 80%

# Combine plots for a comprehensive view
p_quality_combined <- plot_grid(p_completeness, p_confidence, ncol = 1, labels = 'auto')

p_quality_combined

Summarise your data

At any stage you can produce summary statistics (mean, median, skew etc) for your df

op_summarise(df)

Visualise Time Series

Plot the movement of specific keypoints over time to identify noise or gaps.

# Plot the x and y coordinates for Nose (1) and Wrists (4, 7)
p_timeseries <- op_plot_timeseries(df, 
                                   free_y = TRUE, # Allow y-axes to have different scales
                                   overlay = TRUE, # Overlay x and y on the same plot
                                   max_facets = 20) 

p_timeseries

Interpolate missing data

df_interp <- op_interpolate(df, 
                            confidence_threshold = .3, # Set a confidence threshold of 80%
                            treat_na_conf_as_low = TRUE,
                            handle_zeros = TRUE,
                            handle_missing = TRUE)

p_timeseries_interp <- op_plot_timeseries(df_interp, 
                                   free_y = TRUE, # Allow y-axes to have different scales
                                   overlay = TRUE, # Overlay x and y on the same plot
                                   max_facets = 20) 

p_timeseries_interp

Smooth Time Series Data

Use a moving average to smooth the data, which is often necessary before computing velocity or acceleration. Here, we smooth the x-coordinate of the Neck keypoint as an example.

# Apply moving-average smoothing and plot the result
op_smooth_timeseries(df, 
                     keypoints = c('x1'), # Keypoint '1' is the Neck
                     method = 'rollmean',
                     rollmean_width = 5, # 5-frame rolling window
                     plot = TRUE,
                     side = 'right')

---

5. Kinematic Analysis

Now we can compute kinematic variables like velocity, acceleration, and jerk.

Compute Velocity

Velocity is the rate of change of position. We compute it for all keypoints.

# Display a sample of the velocity data
df_velocity %>%
  filter(frame > 1 & frame < 6) %>%
  select(person, frame, region, v_1, v_4, v_7) %>% # Select Neck and Wrists
  nice_table()

Error in `select()`:
! Can't select columns that don't exist.
✖ Column `v_1` doesn't exist.
Run `]8;;x-r-run:rlang::last_trace()rlang::last_trace()]8;;` to see where the error occurred.

Compute Acceleration

Acceleration is the rate of change of velocity.

# We use the velocity dataframe as input
df_acceleration <- op_compute_acceleration(df_velocity, 
                                           merge_xy = TRUE,
                                           overwrite = FALSE,
                                           fps = 30)

# Display a sample of the velocity data
df_acceleration %>%
  filter(frame > 1 & frame < 6) %>%
  select(person, frame, region, a_1, a_4, a_7) %>% # Select Neck and Wrists
  nice_table()

Error in `select()`:
! Can't select columns that don't exist.
✖ Column `a_1` doesn't exist.
Run `]8;;x-r-run:rlang::last_trace()rlang::last_trace()]8;;` to see where the error occurred.

Compute Jerk

Jerk (or jerkiness) is the rate of change of acceleration, often used as a measure of movement smoothness.

base_filename	frame	region	person	v_Nose	v_Neck	v_RShoulder	v_RElbow	v_RWrist	v_LShoulder	v_LElbow	v_LWrist	v_MidHip	v_RHip	v_RKnee	v_RAnkle	v_LHip	v_LKnee	v_LAnkle	v_REye	v_LEye	v_REar	v_LEar	v_LBigToe	v_LSmallToe	v_LHeel	v_RBigToe	v_RSmallToe	v_RHeel	a_Nose	a_Neck	a_RShoulder	a_RElbow	a_RWrist	a_LShoulder	a_LElbow	a_LWrist	a_MidHip	a_RHip	a_RKnee	a_RAnkle	a_LHip	a_LKnee	a_LAnkle	a_REye	a_LEye	a_REar	a_LEar	a_LBigToe	a_LSmallToe	a_LHeel	a_RBigToe	a_RSmallToe	a_RHeel	j_Nose	j_Neck	j_RShoulder	j_RElbow	j_RWrist	j_LShoulder	j_LElbow	j_LWrist	j_MidHip	j_RHip	j_RKnee	j_RAnkle	j_LHip	j_LKnee	j_LAnkle	j_REye	j_LEye	j_REar	j_LEar	j_LBigToe	j_LSmallToe	j_LHeel	j_RBigToe	j_RSmallToe	j_RHeel
Dyad-10.0_B_IDs-B013-B016	2.00	body	left	1.21	0.72	0.60	0.49	87.18	1.01	82.62	85.78	1.41	2.34	87.25	0.00	1.78	121.06	0.00	0.91	82.30	0.45	1.40	0.00	0.00	0.00	0.00	0.00	0.00
Dyad-10.0_B_IDs-B013-B016	3.00	body	left	0.79	84.30	0.98	0.27	0.60	1.88	0.19	0.64	1.02	1.42	1.39	0.00	0.79	85.86	0.00	0.75	1.06	84.96	1.15	0.00	0.00	0.00	0.00	0.00	0.00
Dyad-10.0_B_IDs-B013-B016	4.00	body	left	82.15	0.45	1.14	0.13	124.38	0.47	0.95	125.36	82.86	82.89	0.36	0.00	83.94	121.16	0.00	84.42	84.37	0.79	82.02	0.00	0.00	0.00	0.00	0.00	0.00	2,449.03	2,533.71	17.86	12.01	3,743.10	43.99	27.42	3,779.84	2,469.84	2,467.75	37.43	0.00	2,503.91	5,775.91	0.00	2,511.93	2,512.83	2,529.91	2,426.57	0.00	0.00	0.00	0.00	0.00	0.00
Dyad-10.0_B_IDs-B013-B016	5.00	body	left	2.67	0.36	0.26	1.15	85.35	0.50	0.03	86.47	83.28	83.91	0.86	0.00	84.27	0.51	0.00	81.25	2.14	2.35	84.48	0.00	0.00	0.00	0.00	0.00	0.00	2,390.43	17.93	32.72	38.45	5,812.82	26.10	27.96	2,598.35	4,984.21	5,004.15	27.91	0.00	5,046.30	3,633.77	0.00	3,466.80	2,479.50	50.43	3,548.01	0.00	0.00	0.00	0.00	0.00	0.00	145,183.22	75,857.54	1,252.52	1,497.71	283,073.53	1,139.13	1,654.95	178,184.35	223,617.91	224,148.51	314.87	0.00	226,504.35	279,091.58	0.00	166,385.28	149,769.48	77,409.17	165,809.13	0.00	0.00	0.00	0.00	0.00	0.00
Dyad-10.0_B_IDs-B013-B016	2.00	body	right	0.97	82.02	85.22	83.70	0.03	83.13	85.21	122.38	85.77	83.79	123.04	0.00	0.49	0.60	0.00	0.30	86.39	1.35	83.92	0.00	0.00	0.00	0.00	0.00	0.00	894,714.73	916,397.08	924,879.22	838,819.33	769,788.12	927,496.05	858,490.16	912,430.98	787,137.68	788,366.07	745,521.55	0.00	797,344.54	672,503.66	0.00	916,333.25	920,394.59	958,649.81	1,350,168.67	0.00	0.00	0.00	0.00	0.00	0.00	53,680,632.98	54,977,173.53	55,332,761.46	50,176,766.05	46,187,137.70	55,648,031.36	51,430,963.95	54,755,948.40	47,225,834.22	47,291,535.14	44,820,155.94	0.00	47,839,361.08	40,344,729.86	0.00	54,894,928.58	55,150,722.93	57,517,757.59	94,151,052.14	0.00	0.00	0.00	0.00	0.00	0.00
Dyad-10.0_B_IDs-B013-B016	3.00	body	right	1.11	0.35	0.67	0.47	0.61	82.53	1.62	85.30	85.77	1.17	2.84	0.00	82.80	82.80	0.00	0.31	85.80	1.26	2.04	0.00	0.00	0.00	0.00	0.00	0.00	39.73	2,455.22	2,538.48	2,502.00	18.20	4,969.80	2,548.72	5,781.79	5,146.27	2,536.60	3,744.94	0.00	2,475.17	2,485.05	0.00	3.60	3,597.71	19.39	2,463.21	0.00	0.00	0.00	0.00	0.00	0.00	26,841,093.39	27,497,578.39	27,822,205.89	25,239,483.72	23,094,167.96	27,829,502.08	25,830,992.34	27,444,591.52	23,617,951.30	23,654,593.06	22,277,543.57	0.00	23,846,083.76	20,168,284.19	0.00	27,489,989.71	27,683,459.81	28,758,979.27	40,518,940.73	0.00	0.00	0.00	0.00	0.00	0.00
Dyad-10.0_B_IDs-B013-B016	4.00	body	right	1.38	81.81	1.50	0.48	0.30	0.99	0.42	0.64	85.77	0.89	2.66	0.00	83.10	0.18	0.00	82.52	2.71	2.55	0.31	0.00	0.00	0.00	0.00	0.00	0.00	60.08	2,459.77	59.57	26.76	27.13	2,446.20	59.49	2,565.57	5,146.23	50.98	164.79	0.00	4,977.09	2,478.62	0.00	2,484.61	2,538.48	114.35	60.35	0.00	0.00	0.00	0.00	0.00	0.00	2,992.13	301.87	75,330.08	74,795.31	1,359.69	222,480.00	76,003.87	244,953.91	308,774.95	77,540.04	115,558.60	0.00	223,566.89	148,908.92	0.00	74,540.66	170,850.20	3,542.66	72,184.91	0.00	0.00	0.00	0.00	0.00	0.00
Dyad-10.0_B_IDs-B013-B016	5.00	body	right	116.89	1.12	86.25	0.09	0.21	0.31	84.30	1.20	0.60	0.52	0.67	0.00	83.10	83.16	0.00	1.61	86.94	84.95	83.40	0.00	0.00	0.00	0.00	0.00	0.00	3,465.30	2,421.90	2,543.40	17.10	11.53	28.46	2,520.12	54.48	2,573.16	37.80	98.85	0.00	4,986.00	2,500.22	0.00	2,439.08	2,535.70	2,480.47	2,493.02	0.00	0.00	0.00	0.00	0.00	0.00	102,504.54	146,449.00	74,713.39	1,297.41	1,088.40	72,576.50	75,099.32	77,233.99	231,581.52	2,621.23	7,904.13	0.00	298,891.58	843.51	0.00	147,707.00	105,616.41	71,396.17	75,356.49	0.00	0.00	0.00	0.00	0.00	0.00

Average kinematics overtime for analysis

---

6. Motion Energy Analysis

Motion energy is a measure of the total amount of movement, calculated from the velocity of all keypoints.

# We use the 'df_jerk' dataframe as it contains all previously computed columns
# The function uses the velocity columns ('v_*') to compute motion energy
df_motion <- op_compute_motionenergy(df, 
                                     plot = TRUE)

# The resulting dataframe 'df_motion' now has a 'motion_energy' column
head(select(df_motion, frame, person, motion_energy))

---

7. Synchrony Analysis (Example for Dyadic Data)

Coherence analysis measures the synchrony between two time series (e.g., the motion energy of two people). It is therefore typically used for dyadic data. For this single-person tutorial, we will create a dummy second person’s data just to demonstrate the function’s syntax.

# Run coherance analysis for a single dyad
coherence_results <- op_compute_coherence(df_motion, plot = TRUE)

# View the coherence results summary
coherence_results

---

Session Information

This chunk documents the R version and loaded packages, which is crucial for reproducibility.

# Document your computational environment
sessionInfo()